A Bluffers’ Guide to Iterative Methods for Systems of Linear Equations

The linear systems that are most amenable to solution by iterative methods are those that come from discretisation of partial differential equations, in particular those from mathematical physics. Here are a few of the possible sources of linear systems: • Elliptic equations. These always give a linear system to solve. One characteristic1 of elliptic systems is that all points of the solution are ‘globally coupled’: any point depends on the solution in any other point. This has the immediate consequence that the more parallel a solution method is, the slower it will converge because of the decreased global coupling. • Parabolic equations. These give an elliptic linear system if an implicit method, such as backward Euler, is used. • Nonlinear problems. The Jacobian system in a Newton-Raphson method can be solved iteratively. Iterative methods are particularly attractive here because full precision is often not needed, and one can stop iterating after the desired precision has been reached. • Eigenvalue calculations. In order to find interior eigenvalues of a matrix’ spectrum, one often applies a Lanczos method to a shifted and inverted form of A: in effect one computes eigenvalues of (A−σ)−1. Since a Lanczos method requires multiplication with the coefficient matrix, we need solve a system with A−σ in each iteration. This can be done iteratively, although these systems are by their very nature indefinite, which makes applying iterative methods harder.